The Wiley Blackwell Handbook of Social Anxiety Disorder by Weeks Justin W

Author:Weeks, Justin W.
Language: eng
Format: epub
ISBN: 9781118653890
Publisher: Wiley
Published: 2014-01-31T05:00:00+00:00

A Guide to Evaluating Assessment Tools

A good starting point for discussing the evaluation of assessment tools is the Standards for Educational and Psychological Testing (SEPT; American Educational Research Association, 1999). The SEPT was developed jointly by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education, and consists of a series of guidelines for conceptualizing test construction, evaluation, documentation, fairness, and applications. The SEPT clearly defines reliability and validity, and provides standards for how researchers should use these psychometric terms; we will use the definitions in the SEPT as a starting point for our discussion on reliability and validity.

Reliability

An important consideration when choosing an assessment tool is the degree to which the assessment tool is a reliable method of assessing a particular construct. According to the SEPT, reliability can be defined as “the consistency of [such] measurements when the testing procedure is repeated on a population of individuals or groups” (American Educational Research Association, 1999, p. 25). In other words, reliability simply refers to whether or not an assessment tool consistently produces equivalent results under similar conditions. Below we will review two major reliability-related concepts relevant to testing measures of psychological constructs; although these concepts may be familiar, we will note frequent misunderstandings of their meaning.

Internal consistency

Internal consistency basically refers to the degree to which items within a measure, or subscale, correlate with one another. If five items are thought to assess a particular construct, then these five items should correlate highly with each other. Although showing internal consistency is a good first step in demonstrating reliability, it is not a good estimate of overall reliability for a number of reasons. For example, internal consistency values will overestimate reliability if responses are affected by extraneous commonalities across questions (e.g., the respondent answering all items when in a certain mood even though the measure is not meant to assess mood).

Test-retest reliability

Test-retest reliability refers to the degree to which a measure yields similar scores when administered under similar conditions across time. In other words, a measure with high test-retest reliability, when administered 1 week apart, should yield similar test scores for each respondent. As typically assessed in this literature, test-retest reliability is also a relatively weak assessment of reliability. Typically, test-retest reliability is assessed by repeating an identical measure. Although this is an intuitively compelling test of repeatability, any simple correlation between two administrations is an inextricable combination of true repeatable assessment of the same construct, and various other factors that could lead to higher or lower correlations across time. For example, repeating an identical measure raises, and has no power to dismiss, the possibility that respondents might remember their initial answers and repeat them. These and other concerns about typical test-retest methods are recounted by Nunnally and Bernstein (1994). Ideal tests of reliability are rarely conducted in research related to SAD. Such tests include tests of multiple forms, or at least partially nonoverlapping items, whereas research more often concentrates entirely on predefined measures for which there is only one form.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.